← All Tags
# existential risk → superintelligence

2 links tagged with all of: existential risk + superintelligence

Click any tag below to further narrow down your results

+ long term forecasting (1) + stable endpoints (1) + extrapolation (1) + modeling (1) + pdoom (1) + ai safety (1) + race dynamics (1) + alignment (1)

Links

They really do think AI might kill everyone

AI researchers have spent two decades seriously arguing that superintelligent AI could kill everyone, and this isn't a PR stunt — it's a genuine belief that shapes how they work. The author explains what they think could happen and why they keep building AI anyway.
  • AI researchers use the term "p(doom)" to discuss extinction probability and have been publishing on this since at least 2008, making it an established part of AI safety culture, not a recent panic.
  • Multiple plausible kill mechanisms exist: engineered bioweapons, triggering nuclear war through military AI systems, robot takeover, or unforeseen methods from a superintelligence we can't predict.
  • The "race dynamics" problem: if superintelligent AI is inevitable, being first might be the only way to ensure it's aligned with human values, which creates a perverse incentive to build faster rather than safer.
Last saved Sep 11, 2026 · 7 min read
+ pdoom + ai safety existential risk + race dynamics + alignment superintelligence

Futurism Is Always Extreme

Rigorous models of the distant future inevitably produce extreme outcomes rather than mundane ones, because when you run any consistent logic far enough, the world either reaches a stable endpoint or crashes—and stable endpoints in a post-human future look radically different from today.
  • Qualitative models (like "AI gets smarter faster than humans") can show you the direction of change but not timing, forcing you to extrapolate to logical endpoints that seem implausible.
  • Quantitative models work well for near-term prediction but break down over long timescales, so forecasters must instead ask what stable state a system converges to rather than what happens at a specific date.
  • The only meaningful stable states for civilization are either transformative superintelligence scenarios (Matrioshka brains, uploaded minds, von Neumann probes) or civilizational collapse—there's no middle ground when you follow the logic to its conclusion.
Last saved Aug 30, 2026 · 1 min read
+ long term forecasting + stable endpoints + extrapolation superintelligence existential risk + modeling